Listen to Your Users The Effect of Usability Evaluation on Software Development Practice

A vast majority of the people in the western world use software systems on daily basis for achieving their goals. To be able to do that each person needs to communicate what he or she wants to do to the system and receive a response. This communication needs to be easy for the user, especially when the system is new to him or her. Otherwise, the user either quits using the system; it takes a very long time or gets very irritated. A software team that is making new software needs to evaluate the usability of the system and various methods have been introduced in the literature to do that. My research focus in this thesis is on usability evaluation. I study particularly, how usability evaluation methods can be compared, what data should be gathered in usability evaluation to gain knowledge on how the software affects users who are getting new software for their daily work and how useful this data is to the recipients. Two experiments are reported in this thesis where results from using three different usability evaluation methods are compared. The main result from these two studies is that the think-aloud evaluation method should be used, if the goal of the evaluation is to gather as realistic information as possible on usability problems that the users will have when using the system. Furthermore four case studies are described in the thesis, in which usability evaluation was done by using the think-aloud method in co-operation with real users in their real work situation. These studies give much richer information on the actual use of the systems involved. The findings from one of these case studies indicate that the results from user observation done on a system that users have not seen before or used only for few days are rather similar to the results from usability evaluation done when users have used the system for a longer period. So the common practice of doing user observation on a software system that the participants have not seen before and then interpreting that the results will be the same for actual usage of the system when users will use the system for their real tasks for shorter or longer period is adequate.